Search for: All records

Resilient teams overcome sudden, dynamic changes by enacting rapid, adaptive responses that maintain system effectiveness. We analyzed two experiments on human-autonomy teams (HATs) operating a simulated remotely piloted aircraft system (RPAS) and correlated dynamical measures of resilience with measures of team performance. Across both experiments, HATs experienced automation and autonomy failures, using a Wizard of Oz paradigm. Team performance was measured in multiple ways, using a mission-level performance score, a target processing efficiency score, a failure overcome score, and a ground truth resilience score. Novel dynamical systems metrics of resilience measured the timing of system reorganization in response to failures across RPAS layers, including vehicle, controls, communications layers, and the system overall. Time to achieve extreme values of reorganization and novelty of reorganization were consistently correlated with target processing efficiency and ground truth resilience across both studies. Correlations with mission-level performance and the overcome score were apparent but less consistent. Across both studies, teams displayed greater system reorganization during failures compared to routine task conditions. The second experiment revealed differential effects of team training focused on coordination coaching and trust calibration. These results inform the measurement and training of resilience in HATs using objective, real-time resilience analysis.

 

Urban Search and Rescue (USAR) missions continue to benefit from the incorporation of human–robot teams (HRTs). USAR environments can be ambiguous, hazardous, and unstable. The integration of robot teammates into USAR missions has enabled human teammates to access areas of uncertainty, including hazardous locations. For HRTs to be effective, it is pertinent to understand the factors that influence team effectiveness, such as having shared goals, mutual understanding, and efficient communication. The purpose of our research is to determine how to (1) better establish human trust, (2) identify useful levels of robot transparency and robot explanations, (3) ensure situation awareness, and (4) encourage a bipartisan role amongst teammates. By implementing robot transparency and robot explanations, we found that the driving factors for effective HRTs rely on robot explanations that are context-driven and are readily available to the human teammate. 

The decision process of engaging or disengaging automation has been termed reliance on automation, and it has been widely analyzed as a summary measure of automation usage rather than a dynamic measure. We provide a framework for defining temporal reliance dynamics and apply it to a data-set from a previous study. Our findings show that (1) the higher the reliability of an automated system, the larger the reliance over time; and (2) more workload created by the automation type does not significantly affect the operators’ reliance dynamics in high-reliability systems, but it does produce greater reliance in low-reliability systems. Furthermore, on average, operators with low performance make fewer decision changes and prefer to stick to their decision of using automation even if it is not performing well. Operators with high performance, on average, have a higher frequency of decision change, and therefore, their automation usage periods are shorter. 

Virtual testbeds are fundamental to the success of research on cognitive work in safety-critical domains. A testbed that can meet researchers' objectives and create a sense of reality for participants positively impacts the research process; they have the potential to allow researchers to address questions not achievable in physical environments. This paper discusses the development of a synthetic task environment (STE) for Urban Search and Rescue (USAR) to advance the boundaries of Human-Robot Teams (HRTs) using Roblox. Virtual testbeds can simulate USAR task environments and HRT interactions. After assessing alternative STE platforms, we discovered Roblox not only met our research capabilities but also would prove invaluable for research teams without substantial coding experience. This paper outlines the design process of creating an STE to meet our research team's objectives. 

Risk has been a key factor influencing trust in Human-Automation interactions, though there is no unified tool to study its dynamics. We provide a framework for defining and assessing relative risk of automation usage through performance dynamics and apply this framework to a dataset from a previous study. Our approach allows us to explore how operators’ ability and different automation conditions impact the performance and relative risk dynamics. Our results on performance dynamics show that, on average, operators perform better (1) using automation that is more reliable and (2) using partial automation (more workload) than full automation (less workload). Our analysis of relative risk dynamics indicates that automation with higher reliability has higher relative risk dynamics. This suggests that operators are willing to take more risk for automation with higher reliability. Additionally, when the reliability of automation is lower, operators adapt their behavior to result in lower risk. 

The goal of the Space Challenge project is to identify the challenges faced by teams in space operations and then represent those challenges in a distributed human-machine teaming scenario that resembles typical space operations and to measure the coordination dynamics across the entire system. Currently, several challenges have been identified through semi-structured interviews with nine subject matter experts (SMEs) who were astronauts or those who have experienced or have been involved with interplanetary space exploration. We conducted a thematic analysis on the interviews through an iterative process. Challenges were categorized into four categories, including, communication, training, distributed teaming, and complexity. Based on the findings, challenges and key teamwork characteristics of space operations were integrated into the initial scenario development. In addition to the scenario, we plan to use dynamical system methods to analyze team activity in real time.